Skip to main content

Extending the Derivative: Part 2 (Old Notes)

Previously, we discussed some fundamentals in Multivariable Calculus, including the concept of partial derivatives and the gradient. Now we will discuss some more advanced topics, and as a bonus, apply them to some real-world problems, including Maxwell's Equations.

We will cover the divergence, curl, and Laplacian operators. They are operators in that they take a function as input and return another function. Operators have shown up before, such as the derivative operator , and the gradient operator .

Table of Contents​

Divergence​

Intuitition for Divergence​

Before discussing the formula for divergence, let's first understand what it is. Consider a vector field in 2D space. To drill in the intuition, consider it representing the flow of some water.

Consider a region in the space. In this region, consider how much water is flowing in and out of the region. This is called the divergence of the vector field. For example, set the region to be around the origin. Consider the following vector field:

In this case, the water is flowing out of the region. In this case, the divergence is positive. Note that a positive divergence doesn't mean that all the water is flowing out of the region. It just means that more water is flowing out than in. For example, the divergence here is also positive:

Regions where the divergence is positive are called sources. In the water example, the source could be from a fountain. Next, consider the following vector field:

We flipped the vector field, so now the water is flowing into the region, and the divergence is negative. Regions where the divergence is negative are called sinks. In the water example, the sink could be a drain.

Of course, this is not how water works in real life. Going back to the first example:

Notice that at any region, there's no net flow of water. The water is flowing out of the region as much as it is flowing into the region. Water is incompressible, and the mathematical term for this is that the divergence of the vector field is zero.

Formulation of Divergence​

The notation for divergence we will currently use is (although this is not the only notation). As shown above, divergence is a scalar value, and the operator acts on a vector field.

First, consider a vector field in one dimension:

This means that the water will only flow horizontally. Consider the case where the divergence is positive:

What this means visually is that the water is flowing out of the region more than it is flowing in. On the left side of the region, , and on the right side, .

A visualization is shown below, split into two parts:

  1. The point , and a movable point .
  2. The plot of over (with fixed).

Meaning, it increases as we move to the right. This can be written as:

The second scenario is where there is a lot of water flowing out into the right, and some water flowing in from the left. Recall that the divergence is still positive:

Notice that the slope of the graph is still positive, as it increases as we move to the right.

The last scenario is where there is a lot of water flowing out into the left, and some water flowing in from the right.

The slope of the graph is still positive, as it increases as we move to the right. In this case, it's very negative at the left, and at the right, it is still negative, but less so. Hence,

It should be clear from these scenarios that:

Hence, the partial derivative is somehow related to the divergence. We can apply the same logic for the component of the vector field:

In this case, the water will only flow vertically. Consider the case where the divergence is positive:

Now combine the two components:

The divergence of this vector field is then defined as the sum of the partial derivatives of the components:

There is something interesting about this formula. We only considered vector fields purely in the and directions. In reality, vector fields can be in any direction. The formula for divergence still holds, since the vectors can always be decomposed into components in the and directions. To use the intuitive example, you could consider a box around the origin, and consider the flow of water in and out of the box. Then, the flow can be decomposed into the (left and right of the box) and (up and down of the box) directions.

A small caveat

This isn't actually the formal definition of divergence. The formal definition looks a bit more complicated:

Don't worry about this for now. This is just to show that the concept of divergence is more general than what we've discussed; it is a universal definition for all coordinate systems and dimensions.

Example Problem: Evaluating Divergence​

The vector field is given by:

Evaluate the divergence of .

To evaluate the divergence, we need to find the partial derivatives of the components of the vector field. Recall that partial derivatives are just like normal derivatives, but we treat the other variable as a constant.

First, define the components of the vector field:

Then, find the partial derivatives:

Finally, evaluate the divergence:

Hence, the divergence of is .

Consider some cases:

  • When , the divergence is positive, so the water is flowing out of the region.
  • When , the divergence is negative, so the water is flowing into the region.
  • When , the divergence is zero, so there is no net flow of water.

Nabla Notation for Divergence​

Recall the symbol used for the gradient. The gradient is defined as:

Where we define as:

The divergence is defined as:

This is equivalent to the dot product of the nabla operator and the vector field:

Hence, the divergence can be written as:

Not only does this have symbolic significance, but it also reflects another way to think about divergence intuitively, which will be discussed in a later section.

The First Maxwell Equation: Gauss's Law​

Now that we have discussed divergence, we can apply it to the first Maxwell equation: Gauss's Law.

It describes how the electric field is related to the charge density .

This is a direct proportionality. This means that as the charge density increases, the divergence of the electric field increases, which should make intuitive sense.

Consider a positive charge somewhere in space. The charge density would then be positive, so the divergence of the electric field would be positive. This means that the electric field points away from the charge.

Likewise, for a negative charge, the charge density would be negative, so the divergence of the electric field would be negative.

Divergence in Cylindrical and Spherical Coordinates​

We have discussed divergence in Cartesian coordinates as the sum of the partial derivatives of the components:

Consider a vector field in cylindrical coordinates:

The divergence of this vector field is then the sum of the partial derivatives of the components:

Notice that it is similar to the Cartesian formula, but with some extra things to consider due to the cylindrical coordinates:

  • The component has an extra factor.

The Second Maxwell Equation: Gauss's Law for Magnetism​

The second Maxwell equation describes how the magnetic field is related to the divergence of the magnetic field. It simply states that the divergence of the magnetic field is zero:

Recall that a divergence of zero means that it's incompressible, just like the water example. The magnetic field lines are always closed loops, and they never start or end.

One implication is that there are no magnetic monopoles, hypothetical things that have only a north or south pole.

The first two Maxwell equations are the most intuitive, and they are the most straightforward to understand. They describe in complete detail electro_statics_ and magneto_statics_ - when they are not changing over time. When they do change over time, things get more complicated, and the two fields start to interact with each other. These interactions are described by the last two Maxwell equations.

Curl​

Intuition for Curl in 2D​

Consider a vector field in 2D space. Once again, we will consider it representing the flow of some water.

Focus on this specific region:

Notice that the water is flowing counterclockwise around this point. When there's rotation in the vector field, we call it the curl of the vector field. For counterclockwise rotation, the curl is positive.

Consider another region:

In this region, the water is flowing clockwise around this point, so the curl is negative.

Finally, consider a region where the water does not rotate:

In this case, the curl is zero.

Formulation of Curl in 2D​

The notation for curl is . A nabla notation also exists for curl, which will be discussed later. Since we are considering 2D space, curl behaves a bit differerently, hence we shall use a different notation: .

Decompose the vector field into components:

Consider the case where the curl is positive. This means that the water is rotating counterclockwise. Draw a point and some vectors around it:

Label the signs of the vectors:

Hence, notice that the curl is positive when is decreasing in the direction and is increasing in the direction.

Combining the two components, the curl is defined as:

This also means that does not necessarily have to go from positive to negative, and does not necessarily have to go from negative to positive. In other words, both of these situations contribute the same to the curl:

Example Problem: Evaluating Curl​

The vector field is given by:

Evaluate the curl of .

To evaluate the curl, we need to find the partial derivatives of the components of the vector field. Define the components of the vector field:

Then, find the partial derivatives:

Finally, evaluate the curl:

Hence, the curl of is .

Three-Dimensional Rotation Vectors​

Curl is related to rotation, so it's natural to consider how rotation works in 3D space before discussing the general formula for curl.

First consider 2D rotation. It can be described by a measure of the rotation speed, like 45 degrees per second. For the direction, it depends on the sign of the rotation. Conventionally a positive rotation is counterclockwise. Notice how, just with one number, we can describe 2D rotation in its entirety.

In 3D space, rotation is more complex:

  1. First we need to consider the axis of rotation. This is where the rotation occurs about.
  2. Next, we need to consider the magnitude and direction of the rotation - which is a vector. This is known as the angular velocity, denoted .

The convention for the direction of the angular velocity vector is called the right-hand rule. Take your right hand and curl your fingers in the direction of the rotation. Your thumb then points in the direction of the angular velocity vector. Hence, in a single vector we can describe the axis of rotation, the direction of rotation, and the speed of rotation.

Intuituion for Curl in 3D​

Take the vector field defined previously as:

We have previously plotted the vector field in 2D space. Now, consider putting the same vector field in 3D space, on the plane:

Next, consider the 2D curl of this vector field. Recall that the curl is defined as:

The direction of the curl shall then be in the direction, defined via the right-hand rule. With this, we can plot some vectors representing the curl alongside the vector field:

Notice that in areas of clockwise rotation, the curl is negative (points in the direction), and in areas of counterclockwise rotation, the curl is positive (points in the direction).

The curl is a vector that purely exists in the direction. Hence, a prototype for the 3D curl could be, for a vector field :

However this does not fully capture the curl, since it only works on a 2D vector field. To generalize this more, we need to do this for a vector field in 3D space. Recall our original vector field :

This results, as we've seen, in a vector field that lies on a plane. Consider "copying" this vector for every value:

Of course, only some of the vectors are visible. In reality, for every value, there is a plane in which the vector field exists. Thus, the vector field can be thought of as a volume in 3D space:

We have:

  1. Added a third input to the vector field, . Notice that the output does not depend on , which reflects the fact that the vector field does not change with (i.e. are copies of each other).
  2. Added a third component to the vector field, which is always zero. This reflects how the vectors are all "flat".

Next, just like before, we can add the curl vectors to the vector field. Since the vector field is unchanging in the direction, the curl will also be unchanging in the direction. This means that if you take a column somewhere in the vector field, the curl will be the same for all the vectors in that column.

Recall our prototype for the 3D curl:

We can adjust this to include as an input:

We're not done yet. Recall that is defined such that it is constant in the direction. For vector fields that are not constant in the direction, the curl will be different.

The problem with 3D vector fields is that it is difficult to visualize the direction of rotation. One way to think about it is to consider hovering a ball in the vector field, and letting the fluid flow around it, causing it to rotate.

Formulation of Curl in 3D​

To formulate the curl in 3D, consider a vector field :

Since curl has a direction, we expect the curl to output some 3D vector as well. This formula is usually written in terms of the nabla operator:

Where the nabla operator is defined as . Recall the 2D curl formula from Equation :

We can take our 3D curl formula and treat it as a generalization of the 2D curl formula. Then, the curl in 2D is defined as a special case:

This aligns with our previous definition of curl in 2D in Equation .

For a 3D vector field, recall that the cross product can be defined as the determinant of a certain matrix:

So, the curl of a 3D vector field is hence defined as:

This is an absurd notational trick. We have a matrix of:

  • The unit vectors , , and . Matrices should not have vectors as elements.
  • Partial differential operators.
  • The components of the vector field, which are all multivariable functions.

To compute the determinant, we:

  1. First take and the submatrix formed by removing the row and column of .
  2. Multiply by the determinant of this submatrix.
  3. Repeat this for and .

This results in:

Notice the third term . This is the same as the 2D curl formula, with added to it. This should make conceptual sense.

The Third Maxwell Equation: Faraday's Law​

Faraday's law is one of the four Maxwell equations, which describe how a changing magnetic field induces an electric field. It is given by:

Where is the electric field and is the magnetic field. We can dissect this equation and intuitively understand it.

Consider a magnet on a table. The magnet creates a magnetic field around it, and this field changes when the magnet is moved.

Interpretations of Divergence and Curl with its Formulas​

An interesting way to think about divergence and curl is to consider the formulas for each:

Recall the geometric properties of the dot and cross products:

  • The dot product is the highest when the two vectors are in the same direction.
  • The cross product is the highest when the two vectors are perpendicular.

Recall that the gradient describes how a function changes in space. Consider a point in space, and a step in the direction. Draw a vector for both points:

Consider the difference between the two vectors. In some sense, this represents how the vector field changes with the step. As the divergence increases, then the direction of this difference should be in the direction of the step itself; an outward step should result in an outward difference.

Next, recall that the dot product is the highest when the two vectors are in the same direction. Hence, the dot product of the gradient and the vector field should be the highest when the gradient is in the direction of the vector field. This is the case when the divergence is positive.

Compare this to the curl. The cross product is the highest when the two vectors are perpendicular. As such, the curl should be the highest when the gradient is perpendicular to the vector field.

This is why the curl is highest when the vector field is rotating the fastest.

Laplacian of a Scalar Field​

The Laplacian is another operator for vector fields. It gives a scalar field as an output, and can be thought of as the multivariable generalization of the second derivative. Essentially you add up the second partial derivatives of the components of the scalar field:

We can generalize this to any vector space with basis vectors :

An alternate notation for the Laplacian exists. The Laplacian can be written as the derivative of the first derivative of the vector field:

Hence:

Alternatively, we can use the nabla notation:

Sometimes it is also written using the symbol:

Intuition for Laplacian​

Recall that the divergence is a measure of how much a vector field is expanding or contracting. To illustrate how the Laplacian works, first consider a scalar-valued function :

We can plot this function as a surface in 3D space:

First, we can plot the gradient as a vector field. The gradient points in the direction of the steepest ascent of the function:

Imagine this surface as some sort of mountain. The gradient points in the direction where you would walk to ascend the mountain the fastest.

At some peak of the mountain, the gradient in the points around it would point towards the peak, since that is the steepest ascent. On the other hand, at the "bottom" of the mountain, the gradient would point away from the peak, since that is the steepest descent.

We can view the Laplacian as the divergence of this gradient field. Then:

  1. At the peak, the gradient points towards the peak, and the divergence is positive. So the Laplacian is positive.
  2. At the bottom, the gradient points away from the peak, and the divergence is negative. So the Laplacian is negative.

This is similar to how the second derivative works for single-variable functions:

  1. At a peak, the second derivative is positive, and the function is concave up. This is also called a "local minimum".
  2. At a bottom, the second derivative is negative, and the function is concave down. This is also called a "local maximum".

Example Problem: Laplacian of a Scalar Field​

Evaluate the Laplacian for the following scalar field:

Recall the Laplacian formula:

First, we need to find the gradient of the scalar field, which is a vector of its partial derivatives:

Next, we need to find the divergence of the gradient:

Hence, the Laplacian of the scalar field is:

Harmonic Functions​

Harmonic functions are a special class of functions that satisfy something known as the Laplace equation. It is defined based on the Laplacian of a scalar field:

Since the Laplacian is the multivariable generalization of the second derivative, we can understand this by first taking a single-variable case:

If we integrate this twice, we get:

Meaning it is a straight line. This should also make sense intuitively; a zero second derivative means that the slope is constant throughout the function, so the function should never curve.

Consider the following scalar field:

With a constant value, the function is simply an exponential, and with a nonconstant value, the function is an exponential that sort of oscillates.

Consider the Laplacion of this scalar field:

If we plot the scalar field , it doesn't look like a straight line, but it is a harmonic function:

To show why this is harmonic, consider the single-variable case again:

In a point where , the function is concave up. This means that the points around it are higher than the point itself. Even in a situation where the function has a positive slope:

While there are points that are lower than the point itself, on average, the points around it are higher.

Next, generalize this to a scalar field. If the Laplacian is nonzero, then it means that around the point, its neighbors are higher/lower than the point itself on average:

  1. If the Laplacian is positive (), then the neighbors are higher than the point itself on average.
  2. If the Laplacian is negative (), then the neighbors are lower than the point itself on average.

The function we have is harmonic; the Laplacian is zero. This means that on average, the point's neighbors are at the same height as the point itself.

Laplacians come up in many areas of physics, and this concept of the neighbors' height does have physical significance. For example, if you were considering heat flow, then you would consider the heat around the point to evaluate how the heat is distributed. If the temperature around is higher than the point, then the point will heat up, and vice versa. This can be written as a proportional relationship. Denoting as the temperature at a point at time , then:

The word equation for that is as follows:

This is also known as the heat equation.

Jacobian and Local Linearity​

The Jacobian is another operator for vector fields.

Recall from linear algebra that a matrix can be used to represent a linear transformation. Each column decides where the basis vectors go, and the matrix is hence the result of applying the transformation to the basis vectors.

The Jacobian is a generalization of this concept to multivariable functions.

First, consider a vector field :

This is not a linear transformation. It does not satisfy the properties of a linear transformation, like additivity and homogeneity. We can view this transformation on a grid:

Using the visualization, we can also tell that it is not a linear transformation:

  1. The grid lines are not straight after the transformation.
  2. The grid lines are not parallel and evenly spaced after the transformation.

This function has a special property known as local linearity. Consider a region in the same vector field at a point and zoom in:

Around this point, notice that while the lines still curve a bit, it is almost a linear transformation. We can do even better by zooming in further:

The question is, if the function approaches a linear transformation as we zoom in, what is this linear transformation?

The Jacobian Matrix​

Consider a small step in the direction at the point . This corresponds to some small step in the output vector field after the transformation.

Treat the individual components of the vector field as functions:

Looking at the visualization above, we can see that the step in the output vector field can be expressed as the sum of the steps in the individual components.

Denote the change in as and the change in the vector field as . We can write that is the sum of the changes in the individual components:

Then, the change in the vector field with respect to the change in is:

We can do the same for the change in :

This represents the change in the vector field with respect to the change in :

Combining the vectors of the changes in the vector field with respect to and , we get:

This represents the linear transformation of the vector field around the point , and is known as the Jacobian matrix.

This is very similar to how linear approximation works for single-variable functions. For a single-variable function , if we zoom in around a point , then the function can be approximated as a line. Then the slope of that line (the transformation) can be found by taking a small step in .

The Jacobian can be denoted as . Then, the Jacobian matrix can be generalized to any dimension:

Example Problem: Evaluating the Jacobian Matrix​

Evaluate the Jacobian matrix for the following vector field:

at the point .

Recall the Jacobian matrix is:

First, find the individual components of the vector field:

Next, find the partial derivatives of the components:

Hence, the Jacobian matrix is:

At the point , the Jacobian matrix is then:

The Jacobian Determinant​

The Jacobian matrix is a square matrix, and hence it has a determinant.

What is a Determinant?

At the time of writing, the notes for the determinant of a matrix have not been written yet. Hence, a small introduction to the determinant is given here.

Consider performing a linear transformation on the grid. Draw a square on the grid, and apply the transformation to it:

The determinant measures how much the area of the square changes after the transformation.

Without going into the details, the determinant of a 2x2 matrix is given by:

Let's go back to our example vector field. At the point , we can draw a square around the point and apply the transformation:

Recall the Jacobian matrix for this vector field:

Then, the Jacobian determinant is:

At the point , the Jacobian determinant is then:

This means that the area of the square around the point increases by a factor of 1.224 after the transformation.

The Jacobian determinant is a useful quantity in many areas of mathematics and physics. For example, the Jacobian determinant is used in fluid dynamics to measure how much the volume of a fluid element changes after a coordinate transformation.

Summary​

We have covered a lot of advanced topics related to multivariable derivatives in this chapter. We have introduced these concepts:

  • Divergence: A measure of how much a vector field is expanding or contracting.

  • Curl: A measure of how much a vector field is rotating.

    It is positive when the vector field is rotating counterclockwise. The direction is given by the right-hand rule.

  • Laplacian: A measure of how much a scalar field is curving, a multivariable generalization of the second derivative.

  • Harmonic Functions: Functions that satisfy the Laplace equation, . These functions have the property that the neighbors are at the same height as the point itself on average.

  • Jacobian Matrix: A matrix that represents the linear transformation of a vector field around a point.

    The Jacobian determinant measures how much the area of a square changes after the transformation.